EDET: Entity Descriptor Encoder of Transformer for Multi-Modal Knowledge Graph in Scene Parsing

In scene parsing, the model is required to be able process complex multi-modal data such as images and contexts in real scenes, discover their implicit connections from objects existing scene. As a storage method that contains entity information relationship between entities, knowledge graph can well express semantic this paper, new multi-phase was proposed solve parsing tasks; first, used align then graph-based generates results. We also designed an experiment of feature engineering’s validation for deep-learning preliminarily verify effectiveness method. Hence, we representation named Entity Descriptor Encoder Transformer (EDET), which uses both itself its internal attributes representation. This embedded into transformer structure tasks. EDET aggregate results generation image captioning tasks prove has excellent performance fields. Finally, applied industrial scene, confirmed viability our